archived/identify_key_insights_from_textual_document/document_question_answering.ipynb (1,930 lines of code) (raw):
{
"cells": [
{
"cell_type": "markdown",
"id": "377ca901",
"metadata": {},
"source": [
"# Document Understanding Solution - Question Answering\n"
]
},
{
"cell_type": "markdown",
"id": "60529ae8",
"metadata": {},
"source": [
"---\n",
"\n",
"This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
"\n",
"\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "4c293c47",
"metadata": {},
"source": [
"\n",
"Question Answering is useful when you want to query a large amount of text for specific information. Maybe you're interested in extracting the\n",
"date a certain event happened. You can construct a question (or query) in natural language to retrive this information: e.g. 'When did Company X release Product Y?\". Similar to extractive summarization we saw in the last notebook, Question Answering returns a verbatim slice of the\n",
"text as the answer. It won't generate new words to answer the question. In this notebook, we demonstrate three use cases of Questions and Answering:\n",
"\n",
"1. How to directly deploy a pretrained Transformer-based extractive question answering model to perform inference.\n",
"2. How to fine-tune a pre-trained Transformer model on a custom dataset, and then run inference on the fine-tuned model.\n",
"3. How to run [SageMaker Automatic Model Tuning](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html) (a hyperparameter optimization procedure) to find the best model compared with the model fine-tuned in point 2. The performance of the optimal model and model fine-tuned in point 2 is evaluated on a hold-out test data. \n",
"\n",
"**Note**: When running this notebook on SageMaker Studio, you should make\n",
"sure the `PyTorch 1.10 Python 3.8 CPU Optimized` image/kernel is used. When\n",
"running this notebook on SageMaker Notebook Instance, you should make\n",
"sure the 'sagemaker-soln' kernel is used."
]
},
{
"cell_type": "markdown",
"id": "66f681d6",
"metadata": {},
"source": [
"## 1. Set Up\n",
"\n",
"Before executing the notebook, there are some initial steps required for setup. This notebook requires latest version of sagemaker and ipywidgets."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "90bb2559",
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
},
"pycharm": {
"name": "#%%\n"
},
"tags": []
},
"outputs": [],
"source": [
"!pip install -U sagemaker ipywidgets"
]
},
{
"cell_type": "markdown",
"id": "19300457",
"metadata": {},
"source": [
"We start by importing a variety of packages that are used throughout\n",
"the notebook. One of the most important packages is the Amazon SageMaker\n",
"Python SDK (i.e. `import sagemaker`). We also import modules from our own\n",
"custom (and editable) package that can be found at `../package`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3427447e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import sys\n",
"import boto3\n",
"import json\n",
"import sagemaker\n",
"from sagemaker.pytorch import PyTorchModel\n",
"from sagemaker.local import LocalSession\n",
"import config\n",
"import datetime\n",
"\n",
"IAM_ROLE = sagemaker.get_execution_role()\n",
"aws_region = boto3.Session().region_name\n",
"sess = sagemaker.Session()\n",
"DEFAULT_BUCKET = sess.default_bucket()"
]
},
{
"cell_type": "markdown",
"id": "9a4a5814",
"metadata": {},
"source": [
"Up next, we define the current folder and create a SageMaker client (from\n",
"`boto3`). We can use the SageMaker client to call SageMaker APIs\n",
"directly, as an alternative to using the Amazon SageMaker SDK. We use\n",
"it at the end of the notebook to delete certain resources that are\n",
"created in this notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "173a00dc",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"current_folder = config.get_current_folder(globals())\n",
"sagemaker_client = boto3.client(\"sagemaker\")"
]
},
{
"cell_type": "markdown",
"id": "33a4f8dd",
"metadata": {},
"source": [
"## 2. Run inference on the pre-trained extractive question answering model"
]
},
{
"cell_type": "markdown",
"id": "3df0c5d0",
"metadata": {},
"source": [
"Our question answering system needs a machine learning model. In this\n",
"section, we deploy a model to an Amazon SageMaker Endpoint and then\n",
"invoke the endpoint from the notebook. We use a pre-trained model from\n",
"the [transformers](https://huggingface.co/transformers/) library instead\n",
"of training a model from scratch, specifically the BERT Large model that\n",
"has been pre-trained on the SQuAD dataset.\n",
"\n",
"We use the unique solution prefix to name the model and endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0ec3d8e7",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import uuid\n",
"\n",
"unique_hash = str(uuid.uuid4())[:6]\n",
"endpoint_name = f\"{config.SOLUTION_PREFIX}-{unique_hash}-question-answering-endpoint\""
]
},
{
"cell_type": "markdown",
"id": "9223c9ee",
"metadata": {},
"source": [
"### 2.1. Deploy an endpoint"
]
},
{
"cell_type": "markdown",
"id": "50c54fa2",
"metadata": {},
"source": [
"Up next, we need to define the Amazon SageMaker Model which references\n",
"the source code and the specifies which container to use. \n",
"\n",
"Our pre-trained model is Extractive Question Answering model [bert-large-uncased-whole-word-masking-finetuned-squad](https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad) built on a Transformer model from Hugging Face. It takes two strings as inputs: the first string is a question and the second string is the context or any text you want to use to find the answer of the question, and it returns a sub-string from the context as an answer to the question.\n",
"\n",
"We use the PyTorchModel from the Amazon SageMaker Python SDK. Using PyTorchModel and setting the `framework_version` argument, means that our deployed model runs inside a container that has PyTorch pre-installed (i.e., downloading the model on the fly). Other requirements can be installed by defining a `requirements.txt` file at the specified source_dir location. We use the `entry_point` argument to reference the code (within `source_dir`) that should be run for model inference: functions called `model_fn`, `input_fn`, `predict_fn` and `output_fn` are expected to be defined. And lastly, you can pass `model_data` from a training job, but we are going to load the pre-trained model in the source code running on the endpoint. We still\n",
"need to provide `model_data`, so we pass an empty archive."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "25084a69",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"model = PyTorchModel(\n",
" model_data=f\"{config.SOURCE_S3_PATH}/artifacts/models/empty.tar.gz\",\n",
" entry_point=\"entry_point.py\",\n",
" source_dir=\"containers/question_answering\",\n",
" role=IAM_ROLE,\n",
" framework_version=\"1.5.0\",\n",
" py_version=\"py3\",\n",
" code_location=\"s3://\" + DEFAULT_BUCKET + \"/code\",\n",
" env={\n",
" \"MODEL_ASSETS_S3_BUCKET\": config.SOURCE_S3_BUCKET,\n",
" \"MODEL_ASSETS_S3_PREFIX\": f\"{config.SOURCE_S3_PREFIX}/artifacts/models/question_answering/\",\n",
" \"MMS_DEFAULT_RESPONSE_TIMEOUT\": \"3000\",\n",
" },\n",
")"
]
},
{
"cell_type": "markdown",
"id": "612c847a",
"metadata": {},
"source": [
"Using this Amazon SageMaker Model, we can deploy a HTTPS endpoint on a\n",
"dedicated instance. We choose to deploy the endpoint on a single\n",
"ml.p3.2xlarge instance (or ml.g4dn.2xlarge if unavailable in this\n",
"region). Our question answering model is transfomer that\n",
"benefits from GPU optimization, and a ml.p3.2xlarge has a high\n",
"performance NVIDIA V100 GPU that can reduce inference latency on each\n",
"request. You can expect this deployment step to take around 5 minutes.\n",
"After approximately 15 dashes, you can expect to see an exclamation mark\n",
"which indicates a successful deployment."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "caae5446",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import time\n",
"from sagemaker.serializers import JSONSerializer\n",
"from sagemaker.deserializers import JSONDeserializer\n",
"\n",
"predictor = model.deploy(\n",
" endpoint_name=endpoint_name,\n",
" instance_type=config.HOSTING_INSTANCE_TYPE,\n",
" initial_instance_count=1,\n",
" serializer=JSONSerializer(),\n",
" deserializer=JSONDeserializer(),\n",
")\n",
"\n",
"time.sleep(10)"
]
},
{
"cell_type": "markdown",
"id": "3167342a",
"metadata": {},
"source": [
"When you're trying to update the model for development purposes, but\n",
"experiencing issues because the model/endpoint-config/endpoint already\n",
"exists, you can delete the existing model/endpoint-config/endpoint by\n",
"uncommenting and running the following commands:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "81afaa87",
"metadata": {
"lines_to_next_cell": 2,
"tags": []
},
"outputs": [],
"source": [
"# sagemaker_client.delete_endpoint(EndpointName=endpoint_name)\n",
"# sagemaker_client.delete_endpoint_config(EndpointConfigName=endpoint_name)"
]
},
{
"cell_type": "markdown",
"id": "c360907b",
"metadata": {},
"source": [
"When calling our new endpoint from the notebook, we use a Amazon\n",
"SageMaker SDK\n",
"[`Predictor`](https://sagemaker.readthedocs.io/en/stable/predictors.html).\n",
"A `Predictor` is used to send data to an endpoint (as part of a request),\n",
"and interpret the response. Our `model.deploy` command returned a\n",
"`Predictor` but, by default, it sends and receive numpy arrays. Our\n",
"endpoint expects to receive (and also sends) JSON formatted objects, so\n",
"we modify the `Predictor` to use JSON instead of the PyTorch endpoint\n",
"default of numpy arrays. JSON is used here because it is a standard\n",
"endpoint format and the endpoint response can contain nested data\n",
"structures."
]
},
{
"cell_type": "markdown",
"id": "d8f4b86f",
"metadata": {},
"source": [
"### 2.2. Example input sentences for inference & Query endpoint"
]
},
{
"cell_type": "markdown",
"id": "2057320c",
"metadata": {},
"source": [
"With our model successfully deployed and our predictor configured, we can\n",
"try out the question answering model out on example inputs. All we need\n",
"to do is construct a dictionary object with two keys. `context` is the\n",
"text that we wish to retrieve information from. `question` is the natural\n",
"language query which specifices what information we're interested in\n",
"extracting. We call `predict` on our predictor and we should get a\n",
"response from the endpoint that contains the most likely answers."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ebe8c8a7",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"data = {\"question\": \"what is my name?\", \"context\": \"my name is thom\"}\n",
"response = predictor.predict(data=data)"
]
},
{
"cell_type": "markdown",
"id": "e1fbebc1",
"metadata": {},
"source": [
"We have the responce and we can print out the most likely answers that\n",
"has been extracted from the text above. You'll see each answer has a\n",
"confidence score used for ranking (but this score shouldn't be\n",
"interpreted as a true probability). In addition to the verbatim answer,\n",
"you also get the start and end character indexes of the answer from the\n",
"original context."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e1f65fc2",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"print(response[\"answers\"])"
]
},
{
"cell_type": "markdown",
"id": "c48d8bde",
"metadata": {},
"source": [
"You can try more examples above, but note that this model has been\n",
"pretrained on the SQuAD dataset. You may need to fine-tune this model\n",
"with your own question answering data to obtain better results."
]
},
{
"cell_type": "markdown",
"id": "4185f0fb",
"metadata": {},
"source": [
"### 2.3. Clean up the endpoint\n",
"\n",
"When you've finished with the summarization endpoint (and associated\n",
"endpoint-config), make sure that you delete it to avoid accidental\n",
"charges."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "90bb3485",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Delete the SageMaker endpoint and the attached resources\n",
"predictor.delete_model()\n",
"predictor.delete_endpoint()"
]
},
{
"cell_type": "markdown",
"id": "f0b9081d",
"metadata": {},
"source": [
"## 3. Finetune the pre-trained model on a custom dataset"
]
},
{
"cell_type": "markdown",
"id": "c488afb8",
"metadata": {},
"source": [
"Previously, we saw how to run inference on a pre-trained extractive qusetion answering model. Next, we discuss how a model can be finetuned to a custom dataset. \n",
"\n",
"The Text Embedding model can be fine-tuned on any extractive question \n",
"answering dataset in the same way the model available for inference has been \n",
"fine-tuned on the SQuAD2.0 dataset.\n",
"The model available for fine-tuning attaches an answer extracting layer to the Text Embedding model\n",
"and initializes the layer parameters to random values. The fine-tuning step fine-tunes \n",
"all the model parameters to minimize prediction error on the input data and returns the fine-tuned model.\n",
"The model returned by fine-tuning can be further deployed for inference. Below are the instructions \n",
"for how the training data should be formatted for input to the model. \n",
"\n",
"- **Input:** A directory containing a 'data.csv' file.\n",
" - The first column of the 'data.csv' should have a question.\n",
" - The second column should have the corresponding context.\n",
" - The third column should have the integer character starting position for the answer in the context.\n",
" - The fourth column should have the integer character ending position for the answer in the context.\n",
"- **Output:** A trained model that can be deployed for inference. \n",
" \n",
"Below is an example of 'data.csv' file showing values in its first four columns. Note that the file should not have any header.\n",
"\n",
"| | | | |\n",
"|---|---|---|---|\n",
"|In what country is Normandy located?|\tThe Normans (Norman: Nourmands; French: Normands; Latin: Normanni) were the people who in the 10th and 11th centuries gave their name to Normandy, a region in France. They were descended from Norse (\"Norman\" comes from \"Norseman\") raiders and pirates from Denmark, Iceland and Norway who, under their leader Rollo, agreed to swear fealty to King Charles III of West Francia. Through generations of assimilation and mixing with the native Frankish and Roman-Gaulish populations, their descendants would gradually merge with the Carolingian-based cultures of West Francia. The distinct cultural and ethnic identity of the Normans emerged initially in the first half of the 10th century, and it continued to evolve over the succeeding centuries.|\t159|\t165\n",
"|... | ... |... | ... |\n",
" \n",
"\n",
"SQuAD2.0 dataset is downloaded from \n",
"[Dataset Homepage](https://rajpurkar.github.io/SQuAD-explorer/). \n",
"[CC BY-SA 4.0 License](https://creativecommons.org/licenses/by-sa/4.0/legalcode)."
]
},
{
"cell_type": "markdown",
"id": "5270ea76",
"metadata": {},
"source": [
"### 3.1. Retrieve JumpStart Training artifacts\n",
"Here, for the selected model, we retrieve the training docker container, the training algorithm source, the pre-trained model, and a python dictionary of the training hyper-parameters that the algorithm accepts with their default values. Note that the `model_version`=\"*\" fetches the latest model. Also, we do need to specify the `training_instance_type` to fetch `train_image_uri`."
]
},
{
"cell_type": "markdown",
"id": "4349b37e",
"metadata": {},
"source": [
"You can continue with the default model id, or can choose a different model from the dropdown generated upon running the next cell."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f9d389d0",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"model_id = \"huggingface-eqa-bert-base-uncased\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e81a664b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# download JumpStart model_manifest file.\n",
"boto3.client(\"s3\").download_file(\n",
" f\"jumpstart-cache-prod-{aws_region}\", \"models_manifest.json\", \"models_manifest.json\"\n",
")\n",
"with open(\"models_manifest.json\", \"rb\") as json_file:\n",
" model_list = json.load(json_file)\n",
"\n",
"# filter-out all the Text Classification models from the manifest list.\n",
"eqa_models_all_versions, eqa_models = [\n",
" model[\"model_id\"] for model in model_list if \"-eqa-\" in model[\"model_id\"]\n",
"], []\n",
"[eqa_models.append(model) for model in eqa_models_all_versions if model not in eqa_models]\n",
"\n",
"print(f\"All the other available extractive question answering models are as below.\\n\")\n",
"for each in eqa_models:\n",
" print(f\"{each}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5803ae2a",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from sagemaker import image_uris, model_uris, script_uris, hyperparameters\n",
"\n",
"model_version = \"*\"\n",
"training_instance_type = config.TRAINING_INSTANCE_TYPE\n",
"\n",
"# Retrieve the docker image\n",
"train_image_uri = image_uris.retrieve(\n",
" region=None,\n",
" framework=None,\n",
" model_id=model_id,\n",
" model_version=model_version,\n",
" image_scope=\"training\",\n",
" instance_type=training_instance_type,\n",
")\n",
"# Retrieve the training script\n",
"train_source_uri = script_uris.retrieve(\n",
" model_id=model_id, model_version=model_version, script_scope=\"training\"\n",
")\n",
"# Retrieve the pre-trained model tarball to further fine-tune\n",
"train_model_uri = model_uris.retrieve(\n",
" model_id=model_id, model_version=model_version, model_scope=\"training\"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "5ddec321",
"metadata": {},
"source": [
"### 3.2. Set Training parameters\n",
"\n",
"Now that we are done with all the setup that is needed, we are ready to fine-tune our extractive question answering model. To begin, let us create a [``sageMaker.estimator.Estimator``](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html) object. This estimator launchs the training job. \n",
"\n",
"There are two kinds of parameters that need to be set for training. \n",
"\n",
"The first one are the parameters for the training job. These include: (i) Training data path. This is S3 folder in which the input data is stored, (ii) Output path: This the s3 folder in which the training output is stored. (iii) Training instance type: This indicates the type of machine on which to run the training. Typically, we use GPU instances for these training. We defined the training instance type above to fetch the correct train_image_uri. \n",
"\n",
"The second set of parameters are algorithm specific training hyper-parameters."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "073613ff",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Sample training data is available in this bucket\n",
"training_data_bucket = f\"jumpstart-cache-prod-{aws_region}\"\n",
"# For a quick demonstration of training we have created a random subset of SQuAD-v2 dataset.\n",
"# For complete QNLI dataset replace \"SQuAD-v2-tiny\" with \"SQuAD-v2\" in the line below.\n",
"training_data_prefix = \"training-datasets/SQuAD-v2-tiny/\"\n",
"\n",
"training_dataset_s3_path = f\"s3://{training_data_bucket}/{training_data_prefix}\"\n",
"\n",
"output_bucket = DEFAULT_BUCKET\n",
"output_prefix = \"EQA\"\n",
"\n",
"s3_output_location = f\"s3://{output_bucket}/{output_prefix}/output\""
]
},
{
"cell_type": "markdown",
"id": "36ad14f3",
"metadata": {},
"source": [
"For algorithm specific hyper-parameters, we start by fetching python dictionary of the training hyper-parameters that the algorithm accepts with their default values. This can then be overridden to custom values."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c7c0b469",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from sagemaker import hyperparameters\n",
"\n",
"# Retrieve the default hyper-parameters for fine-tuning the model\n",
"hyperparameters = hyperparameters.retrieve_default(model_id=model_id, model_version=model_version)\n",
"\n",
"# [Optional] Override default hyperparameters with custom values\n",
"hyperparameters[\"batch-size\"] = \"16\"\n",
"print(hyperparameters)"
]
},
{
"cell_type": "markdown",
"id": "79a771ce",
"metadata": {},
"source": [
"### 3.3. Download, preprocess, and upload the training data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "83ba1a15",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!aws s3 cp --recursive $training_dataset_s3_path data/squad2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1d9ae58e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"data = pd.read_csv(\"data/squad2/data.csv\", header=None)"
]
},
{
"cell_type": "markdown",
"id": "85a790c8",
"metadata": {},
"source": [
"View the first five observations of the training data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c8844443",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"data.head(5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2dd6423f",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from sklearn.model_selection import train_test_split"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59fb5adb",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"train_data, test_data = train_test_split(data, test_size=0.15, random_state=42)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e6dde7f5",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"train_data.to_csv(\"data/squad2/split_train.csv\", header=False, index=False)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a02e44af",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import os\n",
"\n",
"prefix = \"EQA\"\n",
"boto3.Session().resource(\"s3\").Bucket(DEFAULT_BUCKET).Object(\n",
" os.path.join(prefix, \"train/data.csv\")\n",
").upload_file(\"data/squad2/split_train.csv\")"
]
},
{
"cell_type": "markdown",
"id": "5cc35d47",
"metadata": {},
"source": [
"Process the text data to make them ready for inference. In particular, the question with multiple answers are formulated as multiple rows in above `data`. For an example, one question with three answers will yield three rows in the `data`. Each row corresponds to one answer and has the same question content. In evalution, we combine those rows that correspond to the same question. As a result, each input in test examples has an unique question content and its corresponded ground truth answers can be muliple. For each test example for inference, which includes one context and question, the model outputs a predicted answer. Next. we compare the predicted answer with each of the ground truth answers and use the best comparision result for model performance on that example. Details are shown in the inference section as below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b8ed1d27",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"unique_test_examples = {}\n",
"\n",
"for idx, row in test_data.iterrows():\n",
" if row[0] not in unique_test_examples:\n",
" unique_test_examples[row[0]] = {\n",
" \"context\": row[1],\n",
" \"answer\": [row[1][row[2] : row[3]]],\n",
" }\n",
" else:\n",
" assert row[1] == unique_test_examples[row[0]][\"context\"]\n",
" unique_test_examples[row[0]][\"answer\"].append(row[1][row[2] : row[3]])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "06542a56",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"test_examples = []\n",
"ground_truth = []\n",
"for key in unique_test_examples:\n",
" test_examples.append([key, unique_test_examples[key][\"context\"]])\n",
" ground_truth.append(unique_test_examples[key][\"answer\"])"
]
},
{
"cell_type": "markdown",
"id": "7b28f4fb",
"metadata": {},
"source": [
"### 3.4. Fine-tuning without hyperparameter optimization"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c1ecfcde",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from sagemaker.estimator import Estimator\n",
"from sagemaker.utils import name_from_base\n",
"from sagemaker.tuner import HyperparameterTuner\n",
"\n",
"from sagemaker import get_execution_role\n",
"\n",
"role = get_execution_role()\n",
"\n",
"training_job_name = training_job_name = f\"{config.SOLUTION_PREFIX}-eqa-finetune\"\n",
"\n",
"# Create SageMaker Estimator instance\n",
"eqa_estimator = Estimator(\n",
" role=role,\n",
" image_uri=train_image_uri,\n",
" source_dir=train_source_uri,\n",
" model_uri=train_model_uri,\n",
" entry_point=\"transfer_learning.py\",\n",
" instance_count=1,\n",
" instance_type=training_instance_type,\n",
" max_run=360000,\n",
" hyperparameters=hyperparameters,\n",
" output_path=s3_output_location,\n",
" tags=[{\"Key\": config.TAG_KEY, \"Value\": config.SOLUTION_PREFIX}],\n",
" base_job_name=training_job_name,\n",
" debugger_hook_config=False,\n",
")\n",
"\n",
"\n",
"training_data_path_updated = f\"s3://{DEFAULT_BUCKET}/{prefix}/train\"\n",
"# Launch a SageMaker Training job by passing s3 path of the training data\n",
"eqa_estimator.fit({\"training\": training_data_path_updated}, logs=True)"
]
},
{
"cell_type": "markdown",
"id": "1ff98929",
"metadata": {},
"source": [
"## 3.5. Deploy & run Inference on the fine-tuned model\n",
"\n",
"A trained model does nothing on its own. We now want to use the model to perform inference. For this example, that means predicting the answer of an input sentence including a context and question. We follow the same steps as in `2. Run inference on the pre-trained extractive question answering model`. We start by retrieving the jumpstart artifacts for deploying an endpoint. However, instead of base_predictor, we deploy the `eqa_estimator` that we fine-tuned."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ba3cee88",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"inference_instance_type = config.HOSTING_INSTANCE_TYPE\n",
"\n",
"# Retrieve the inference docker container uri\n",
"deploy_image_uri = image_uris.retrieve(\n",
" region=None,\n",
" framework=None,\n",
" image_scope=\"inference\",\n",
" model_id=model_id,\n",
" model_version=model_version,\n",
" instance_type=inference_instance_type,\n",
")\n",
"# Retrieve the inference script uri\n",
"deploy_source_uri = script_uris.retrieve(\n",
" model_id=model_id, model_version=model_version, script_scope=\"inference\"\n",
")\n",
"\n",
"endpoint_name_finetune = (\n",
" f\"{config.SOLUTION_PREFIX}eqa-finetune-{datetime.datetime.now().strftime('%Y-%m-%d-%H%M%S')}\"\n",
")\n",
"\n",
"# Use the estimator from the previous step to deploy to a SageMaker endpoint\n",
"finetuned_predictor = eqa_estimator.deploy(\n",
" initial_instance_count=1,\n",
" instance_type=inference_instance_type,\n",
" entry_point=\"inference.py\",\n",
" image_uri=deploy_image_uri,\n",
" source_dir=deploy_source_uri,\n",
" endpoint_name=endpoint_name_finetune,\n",
")\n",
"\n",
"time.sleep(10)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1416df24",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"newline, bold, unbold = \"\\n\", \"\\033[1m\", \"\\033[0m\"\n",
"\n",
"\n",
"def query_endpoint(encoded_text, predictor):\n",
" response = predictor.predict(\n",
" encoded_text,\n",
" {\"ContentType\": \"application/list-text\", \"Accept\": \"application/json;verbose\"},\n",
" )\n",
" return response\n",
"\n",
"\n",
"def parse_response(query_response):\n",
" model_predictions = json.loads(query_response)\n",
" answer = (model_predictions[\"answer\"],)\n",
" return answer"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "43bad1f7",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"predictions = []\n",
"for question_context in test_examples:\n",
" query_response = query_endpoint(\n",
" json.dumps(question_context).encode(\"utf-8\"), finetuned_predictor\n",
" )\n",
" answer = parse_response(query_response)\n",
" predictions.append(answer[0])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2ae61d33",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# these functions are heavily influenced by the HF squad_metrics.py script\n",
"def normalize_text(s):\n",
" \"\"\"Removing articles and punctuation, and standardizing whitespace are all typical text processing steps.\"\"\"\n",
" import string, re\n",
"\n",
" def remove_articles(text):\n",
" regex = re.compile(r\"\\b(a|an|the)\\b\", re.UNICODE)\n",
" return re.sub(regex, \" \", text)\n",
"\n",
" def white_space_fix(text):\n",
" return \" \".join(text.split())\n",
"\n",
" def remove_punc(text):\n",
" exclude = set(string.punctuation)\n",
" return \"\".join(ch for ch in text if ch not in exclude)\n",
"\n",
" def lower(text):\n",
" return text.lower()\n",
"\n",
" return white_space_fix(remove_articles(remove_punc(lower(s))))\n",
"\n",
"\n",
"def compute_exact_match(prediction, truth):\n",
" return int(normalize_text(prediction) == normalize_text(truth))\n",
"\n",
"\n",
"def compute_f1(prediction, truth):\n",
" pred_tokens = normalize_text(prediction).split()\n",
" truth_tokens = normalize_text(truth).split()\n",
"\n",
" # if either the prediction or the truth is no-answer then f1 = 1 if they agree, 0 otherwise\n",
" if len(pred_tokens) == 0 or len(truth_tokens) == 0:\n",
" return int(pred_tokens == truth_tokens)\n",
"\n",
" common_tokens = set(pred_tokens) & set(truth_tokens)\n",
"\n",
" # if there are no common tokens then f1 = 0\n",
" if len(common_tokens) == 0:\n",
" return 0\n",
"\n",
" prec = len(common_tokens) / len(pred_tokens)\n",
" rec = len(common_tokens) / len(truth_tokens)\n",
"\n",
" return 2 * (prec * rec) / (prec + rec)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3080fd60",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"em_score, f1_score = [], []\n",
"\n",
"for prediction, gold_answers in zip(predictions, ground_truth):\n",
" em_score.append(max((compute_exact_match(prediction, answer)) for answer in gold_answers))\n",
" f1_score.append(max((compute_f1(prediction, answer)) for answer in gold_answers))\n",
"\n",
"print(f\"Average Exact Matching score: {np.mean(em_score)}\")\n",
"print(f\"Average F1 score: {np.mean(f1_score)}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "aaeddcb7",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"result = {\n",
" \"Average Exact Matching score\": [np.mean(em_score)],\n",
" \"Average F1 Score\": [np.mean(f1_score)],\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "495fddf7",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"result = pd.DataFrame.from_dict(result, orient=\"index\", columns=[\"No HPO\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "92aeb2fd",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"result"
]
},
{
"cell_type": "markdown",
"id": "2eff861b",
"metadata": {},
"source": [
"## 4. Finetune the pre-trained model on a custom dataset with automatic model tuning (AMT)\n",
"\n",
"Amazon SageMaker automatic model tuning, also known as hyperparameter tuning, finds the best version of a model by running many training jobs on your dataset using the algorithm and ranges of hyperparameters that you specify. It then chooses the hyperparameter values that result in a model that performs the best, as measured by a metric that you choose. We use a [HyperparameterTuner](https://sagemaker.readthedocs.io/en/stable/api/training/tuner.html) object to interact with Amazon SageMaker hyperparameter tuning APIs."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5d63c93e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from sagemaker.tuner import (\n",
" ContinuousParameter,\n",
" IntegerParameter,\n",
" CategoricalParameter,\n",
" HyperparameterTuner,\n",
")\n",
"\n",
"\n",
"# Define objective metric per framework, based on which the best model is selected.\n",
"metric_definitions_per_model = {\n",
" \"huggingface\": {\n",
" \"metrics\": [{\"Name\": \"val_loss\", \"Regex\": \"'eval_loss': ([0-9]+(.|e\\-)[0-9]+),?\"}],\n",
" \"type\": \"Minimize\",\n",
" }\n",
"}\n",
"\n",
"# You can select from the hyperparameters supported by the model, and configure ranges of values to be searched for training the optimal model.(https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-ranges.html)\n",
"hyperparameter_ranges = {\n",
" \"adam-learning-rate\": ContinuousParameter(0.00001, 0.01, scaling_type=\"Logarithmic\"),\n",
" \"epochs\": IntegerParameter(3, 10),\n",
" \"train-only-top-layer\": CategoricalParameter([\"True\", \"False\"]),\n",
"}\n",
"\n",
"# Increase the total number of training jobs run by AMT, for increased accuracy (and training time).\n",
"max_jobs = 6\n",
"# Change parallel training jobs run by AMT to reduce total training time, constrained by your account limits.\n",
"# if max_jobs=max_parallel_jobs then Bayesian search turns to Random.\n",
"max_parallel_jobs = 3"
]
},
{
"cell_type": "markdown",
"id": "5a066390",
"metadata": {},
"source": [
"### 4.1. Fine-tuning with hyperparameter optimization"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f7ae1e31",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from sagemaker.estimator import Estimator\n",
"from sagemaker.tuner import HyperparameterTuner\n",
"\n",
"tuning_job_name = f\"{config.SOLUTION_PREFIX}-eqa-hpo\"\n",
"\n",
"# Create SageMaker Estimator instance\n",
"eqa_estimator = Estimator(\n",
" role=role,\n",
" image_uri=train_image_uri,\n",
" source_dir=train_source_uri,\n",
" model_uri=train_model_uri,\n",
" entry_point=\"transfer_learning.py\",\n",
" instance_count=1,\n",
" instance_type=training_instance_type,\n",
" max_run=360000,\n",
" hyperparameters=hyperparameters,\n",
" output_path=s3_output_location,\n",
" tags=[{\"Key\": config.TAG_KEY, \"Value\": config.SOLUTION_PREFIX}],\n",
" base_job_name=tuning_job_name,\n",
" debugger_hook_config=False,\n",
")\n",
"\n",
"metric_definitions = next(\n",
" value for key, value in metric_definitions_per_model.items() if model_id.startswith(key)\n",
")\n",
"\n",
"hp_tuner = HyperparameterTuner(\n",
" eqa_estimator,\n",
" metric_definitions[\"metrics\"][0][\"Name\"],\n",
" hyperparameter_ranges,\n",
" metric_definitions[\"metrics\"],\n",
" max_jobs=max_jobs,\n",
" max_parallel_jobs=max_parallel_jobs,\n",
" objective_type=metric_definitions[\"type\"],\n",
" base_tuning_job_name=training_job_name,\n",
")\n",
"\n",
"# Launch a SageMaker Tuning job to search for the best hyperparameters\n",
"hp_tuner.fit({\"training\": training_data_path_updated})"
]
},
{
"cell_type": "markdown",
"id": "406d730f",
"metadata": {},
"source": [
"### 4.2. Deploy & run Inference on the fine-tuned model"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9e295561",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Retrieve the inference docker container uri\n",
"deploy_image_uri = image_uris.retrieve(\n",
" region=None,\n",
" framework=None,\n",
" image_scope=\"inference\",\n",
" model_id=model_id,\n",
" model_version=model_version,\n",
" instance_type=inference_instance_type,\n",
")\n",
"# Retrieve the inference script uri\n",
"deploy_source_uri = script_uris.retrieve(\n",
" model_id=model_id, model_version=model_version, script_scope=\"inference\"\n",
")\n",
"\n",
"endpoint_name_hpo = f\"{config.SOLUTION_PREFIX}-eqa-hpo-endpoint\"\n",
"\n",
"# Use the estimator from the previous step to deploy to a SageMaker endpoint\n",
"finetuned_predictor_hpo = hp_tuner.deploy(\n",
" initial_instance_count=1,\n",
" instance_type=inference_instance_type,\n",
" entry_point=\"inference.py\",\n",
" image_uri=deploy_image_uri,\n",
" source_dir=deploy_source_uri,\n",
" endpoint_name=endpoint_name_hpo,\n",
")\n",
"\n",
"time.sleep(10)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "af277939",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"predictions_hpo = []\n",
"for question_context in test_examples:\n",
" query_response = query_endpoint(\n",
" json.dumps(question_context).encode(\"utf-8\"), finetuned_predictor_hpo\n",
" )\n",
" answer = parse_response(query_response)\n",
" predictions_hpo.append(answer[0])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9f96bd40",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"em_score_hpo, f1_score_hpo = [], []\n",
"\n",
"for prediction, gold_answers in zip(predictions_hpo, ground_truth):\n",
" em_score_hpo.append(max((compute_exact_match(prediction, answer)) for answer in gold_answers))\n",
" f1_score_hpo.append(max((compute_f1(prediction, answer)) for answer in gold_answers))\n",
"\n",
"print(f\"Average Exact Matching score: {np.mean(em_score_hpo)}\")\n",
"print(f\"Average F1 score: {np.mean(f1_score_hpo)}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3ef80820",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"result_hpo = {\n",
" \"Average Exact Matching score\": [np.mean(em_score_hpo)],\n",
" \"Average F1 Score\": [np.mean(f1_score_hpo)],\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2862e520",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"result_hpo = pd.DataFrame.from_dict(result_hpo, orient=\"index\", columns=[\"With HPO\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d8d6728d",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"pd.concat([result, result_hpo], axis=1)"
]
},
{
"cell_type": "markdown",
"id": "f0f97842",
"metadata": {},
"source": [
"We can see results with hyperparameter optimization shows better performance on the hold-out test data."
]
},
{
"cell_type": "markdown",
"id": "f46d2184",
"metadata": {},
"source": [
"## 4.3. Clean Up the endpoint\n",
"\n",
"When you've finished with the summarization endpoint (and associated\n",
"endpoint-config), make sure that you delete it to avoid accidental\n",
"charges."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "82b27421",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Delete the SageMaker endpoint and the attached resources\n",
"finetuned_predictor.delete_model()\n",
"finetuned_predictor.delete_endpoint()\n",
"\n",
"finetuned_predictor_hpo.delete_model()\n",
"finetuned_predictor_hpo.delete_endpoint()"
]
},
{
"cell_type": "markdown",
"id": "7eef31c1",
"metadata": {},
"source": [
"## Next Stage\n",
"\n",
"We've just looked at how you can query document for specific information.\n",
"Up next we look at a technique that can be used to extract the key\n",
"entities from a document, called Entity Recognition.\n",
"\n",
"[Click here to continue with Name Entity Recognition.](./4_entity_recognition.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "e2947b02",
"metadata": {},
"source": [
"## Notebook CI Test Results\n",
"\n",
"This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
}
],
"metadata": {
"availableInstances": [
{
"_defaultOrder": 0,
"_isFastLaunch": true,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 4,
"name": "ml.t3.medium",
"vcpuNum": 2
},
{
"_defaultOrder": 1,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.t3.large",
"vcpuNum": 2
},
{
"_defaultOrder": 2,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.t3.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 3,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.t3.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 4,
"_isFastLaunch": true,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.m5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 5,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.m5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 6,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.m5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 7,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.m5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 8,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.m5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 9,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.m5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 10,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.m5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 11,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.m5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 12,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.m5d.large",
"vcpuNum": 2
},
{
"_defaultOrder": 13,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.m5d.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 14,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.m5d.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 15,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.m5d.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 16,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.m5d.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 17,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.m5d.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 18,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.m5d.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 19,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.m5d.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 20,
"_isFastLaunch": false,
"category": "General purpose",
"gpuNum": 0,
"hideHardwareSpecs": true,
"memoryGiB": 0,
"name": "ml.geospatial.interactive",
"supportedImageNames": [
"sagemaker-geospatial-v1-0"
],
"vcpuNum": 0
},
{
"_defaultOrder": 21,
"_isFastLaunch": true,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 4,
"name": "ml.c5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 22,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 8,
"name": "ml.c5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 23,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.c5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 24,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.c5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 25,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 72,
"name": "ml.c5.9xlarge",
"vcpuNum": 36
},
{
"_defaultOrder": 26,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 96,
"name": "ml.c5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 27,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 144,
"name": "ml.c5.18xlarge",
"vcpuNum": 72
},
{
"_defaultOrder": 28,
"_isFastLaunch": false,
"category": "Compute optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.c5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 29,
"_isFastLaunch": true,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.g4dn.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 30,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.g4dn.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 31,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.g4dn.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 32,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.g4dn.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 33,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.g4dn.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 34,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.g4dn.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 35,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 61,
"name": "ml.p3.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 36,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 244,
"name": "ml.p3.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 37,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 488,
"name": "ml.p3.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 38,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.p3dn.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 39,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.r5.large",
"vcpuNum": 2
},
{
"_defaultOrder": 40,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.r5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 41,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.r5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 42,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.r5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 43,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.r5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 44,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.r5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 45,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 512,
"name": "ml.r5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 46,
"_isFastLaunch": false,
"category": "Memory Optimized",
"gpuNum": 0,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.r5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 47,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 16,
"name": "ml.g5.xlarge",
"vcpuNum": 4
},
{
"_defaultOrder": 48,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 32,
"name": "ml.g5.2xlarge",
"vcpuNum": 8
},
{
"_defaultOrder": 49,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 64,
"name": "ml.g5.4xlarge",
"vcpuNum": 16
},
{
"_defaultOrder": 50,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 128,
"name": "ml.g5.8xlarge",
"vcpuNum": 32
},
{
"_defaultOrder": 51,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 1,
"hideHardwareSpecs": false,
"memoryGiB": 256,
"name": "ml.g5.16xlarge",
"vcpuNum": 64
},
{
"_defaultOrder": 52,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 192,
"name": "ml.g5.12xlarge",
"vcpuNum": 48
},
{
"_defaultOrder": 53,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 4,
"hideHardwareSpecs": false,
"memoryGiB": 384,
"name": "ml.g5.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 54,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 768,
"name": "ml.g5.48xlarge",
"vcpuNum": 192
},
{
"_defaultOrder": 55,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 1152,
"name": "ml.p4d.24xlarge",
"vcpuNum": 96
},
{
"_defaultOrder": 56,
"_isFastLaunch": false,
"category": "Accelerated computing",
"gpuNum": 8,
"hideHardwareSpecs": false,
"memoryGiB": 1152,
"name": "ml.p4de.24xlarge",
"vcpuNum": 96
}
],
"kernelspec": {
"display_name": "Python 3 (Data Science 3.0)",
"language": "python",
"name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-310-v1"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}